The Infiniband® network includes nodes that communicate through a channel-based switched fabric (Infiniband® is a registered trademark of Infiniband Trade Association, located in Beaverton, Oreg.). For example, the nodes may be a host, an input/output subsystem, or a router which connects to another network. The switched fabric is made up of a collection of switches, routers, and/or links that connect a set of channel adapters. The channel adapters form an interface between the switched fabric and the nodes. The channel adapter of the host is referred to as a host channel adapter. The channel adapter of an I/O subsystem is referred to as a target channel adapter.
In Infiniband®, two processes communicate using a queue pair. A queue pair includes a send queue and a receive queue. Specifically, in order for a process to send a message to another process, the process posts the message to the send queue. The host channel adapter sends the message in the form of packets to the channel adapter having the receive queue. Each packet that is sent includes at least an Infiniband® header and a packet payload. The payload is typically stored in on-chip random access memory (RAM). Logic associated with the receive queue ensures that packets are processed in a particular order using packet sequence numbers.
Infiniband® supports operations such as remote direct memory access (RDMA) read and write operations. Specifically, a requesting process may send a request in the message for the receiver to store data and/or return data. In response, the remote node's channel adapter includes functionality to store and/or return the requested data to the requesting process. As with any type of transmission and response system, errors may occur in networking activity, e.g., during packet transmission or reception. When errors occur, dedicated hardware is provided to collect and data for debugging purposes.